Best Practices for A/B Testing Your Business Text Messages

What is SMS A/B testing? 

A/B testing, also known as split testing or split-variant testing, refers to the randomized experimentation process wherein two or more versions of a variable (subject line, content, CTAs) are shown to different audience segments at the same time. This is done to determine which version leaves the maximum impact and drives business metrics. In business texting, this is done mostly to determine which text will drive better engagement.

What are the benefits of A/B testing text messages? 

In essence, A/B testing allows the sender to test different content compositions and variables to inform future decisions based on what performs better with their desired target audience. Senders utilizing A/B testing have been proven to receive better open rates, click through rates, reply rates, and overall engagement.

How can A/B testing be used within business texting? 

A/B testing is most commonly used in email, web design, UX, product design and texting. Within the world of business texting, A/B testing is best utilized to increase engagementby testing message copy, CTAs (calls to action), imagery, link title, and more. 

What are the best practices for A/B testing? 

1. Test the Right Pieces
   In business texting, your number one goal should be audience engagement. Ensure you are testing specific points in copy (i.e passive language vs. direct) so you can get clean data to make smart decisions.
2. Set Your Sample Size
   A/B testing on a group of 10 will almost always provide inconclusive results. When running a test, make sure it is with a larger segment of similar customers so you can be sure to get a clear winner.
3. Do Your Research and Set Goals
   Dirty data is the root of poor decisions. Be sure to lay out your A/B test with set KPIs (key performance indicators) to determine what success looks like in that scenario.
4. Schedule Your Tests Correctly
   Utilize TextUs data to understand your audience segment and the best performing open and reply times of day.
5. Nail Your Test Duration
   Every A/B test will ask what % of your audience you want to send the test to, along with how long the test should run before determining the winner and sending it to the rest of your segment. We suggest testing at least 50% of your audience (split 25/25) then sending the winner to the rest of the audience after at least 4 hours of feedback to allow for best results.
6. Test One Element at a Time
   Never try to test a CTA as well as an image. This will provide inconsistent results and dirty data. By only testing one element at a time, you’ll be able to gather clearer results around the impact of your change.
7. Keep Variations Under Control
   Variations in a text should be limited to smaller changes that will directly affect the desired engagement you are testing for. If you want to see what image will get better results for your audience to send a reply, start with one text that has an image and one that has no image at all. If the text with an image gets more replies, then test what type of image gets a better response and so on.
8. Pay Attention to the Data
   Data doesn’t lie! When in doubt, trust the numbers then test, test, test again!

How does TextUs support A/B testing? 

TextUs offers built-in A/B testing tools for your SMS campaigns with our premium add-on Campaigns Pro. This functionality allows you to easily determine your strongest performing message by A/B testing up to three message variants per campaign. Whether you want to test message performance based on copy, user personalizations, or the addition of rich media, Campaigns Pro allows you to compare performance metrics such as response rates and link clickthrough rates so you can pick the strongest version for future campaigns. 

To take advantage of these tools, simply create a new one-time campaign, select your sample audience, then add up to three message variations to the campaign. TextUs strongly recommends following the best practices listed above when choosing which variations you’d like to test so you can get the best results. Once your campaign is sent, view your campaign results to see which variation was the top performer. Happy with the results? Create a new campaign using your top performing message to drive higher engagement and better response rates with your target audience.

Interested in learning more? Reach out to your TextUs Account Manager or request a demo. 

Already have Campaigns Pro? Click here to learn more about how to set up your A/B testing campaigns in TextUs.

State of SMS Engagement 2024

Looking to increase engagement with your candidates, prospects, and customers in 2024? Look no further! We conducted a survey paired with usage data to figure out which communication channels are most effective for which industry and size. Discover all of the tips about business communication and download our free report today!

Download Now